Image calibration method and device

ABSTRACT

An image calibration method and a device are provided. The method includes: obtaining a pupil position offset of a user and a visual acuity of the user, where the pupil position offset of the user is an offset of a pupil of the user relative to an optical axis; and adjusting a to-be-displayed image based on the pupil position offset of the user and the visual acuity of the user. In this way, when the user wears a head-mounted display device improperly, a point seen by the user when the user looks straight ahead is a center point of an image displayed on a display.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a national stage of International Application No.PCT/CN2021/135168, filed on Dec. 2, 2021, which claims priority toChinese Patent Application No. 202011469253.5, filed on Dec. 14, 2020.Both of the aforementioned applications are hereby incorporated byreference in their entireties.

TECHNICAL FIELD

This application relates to the field of terminal technologies, and inparticular, to an image calibration method and a device.

BACKGROUND

A head-mounted display device includes one left optical system and oneright optical system, and each optical system includes an optical lensand a display. After a user wears the head-mounted display device, theoptical lens refracts light from the display into a human eye, so thatthe user can see a magnified virtual image. This enhances immersion ofthe user.

However, due to a mismatch of a pupillary distance of the user with adistance between two optical axes of the head-mounted display device, orimproper wearing of the device by the user, or simply other reasons,points seen by the user when two eyes of the user look straight aheadare shifted at different degrees relative to center points of imagesdisplayed on displays. Consequently, the two eyes see differentpictures, and the user feels dizzy.

SUMMARY

This application provides an image calibration method and a device, tocalibrate an image.

According to a first aspect, this application provides an imagecalibration method, including: obtaining a pupil position offset of auser and a visual acuity of the user, where the pupil position offset ofthe user is an offset of a pupil of the user relative to an opticalaxis; determining, based on the pupil position offset of the user andthe visual acuity of the user, a first image center point offsetcorresponding to the pupil position offset of the user and the visualacuity of the user; and adjusting a center point of a to-be-displayedimage based on the first image center point offset and a pupil shiftdirection.

In a possible implementation, the determining, based on the pupilposition offset of the user and the visual acuity of the user, a firstimage center point offset corresponding to the pupil position offset ofthe user and the visual acuity of the user includes: determining thefirst image center point offset based on the pupil position offset ofthe user, the visual acuity of the user, and a first mappingrelationship. The first mapping relationship indicates a correspondencebetween a pupil position offset, a visual acuity, and an image centerpoint offset.

In a possible implementation, the adjusting a center point of ato-be-displayed image based on the first image center point offset and apupil shift direction includes: moving the center point of theto-be-displayed image generated by a first module toward the pupil shiftdirection by a first distance. The first distance is the same as thefirst image center point offset, and the first module is configured torender scene data according to an instruction input by the user or adefault playing order, to generate a corresponding image. The firstmodule is also referred to as a virtual camera in embodiments of thisapplication.

In a possible implementation, the adjusting a center point of ato-be-displayed image based on the first image center point offset and apupil shift direction includes: adjusting a displacement parameter valueof a first module based on the first image center point offset, so thatthe center point of the to-be-displayed image generated by the firstmodule is moved toward the pupil shift direction by a first distance.The first distance is the same as the first image center point offset,and the first module is configured to render scene data according to aninstruction input by the user or a default playing order, to generate acorresponding image.

In a possible implementation, the obtaining a pupil position offset of auser includes: obtaining an eye image of the user; recognizing a pupilposition from the eye image; and using a distance between the pupilposition and a reference position as the pupil position offset of theuser.

In a possible implementation, the obtaining a visual acuity of the userincludes: obtaining a first position of each optical lens; and searchinga fourth mapping relationship for a first visual acuity corresponding tothe first position, and using the first visual acuity as the visualacuity of the user. The fourth mapping relationship indicates acorrespondence between a position of each optical lens and a visualacuity.

In the image calibration method, when the user wears a head-mounteddisplay device improperly, a point seen by the user when the user looksstraight ahead is a center point of an image displayed on a display. Inaddition, after the foregoing calibration is performed on imagescorresponding to optical systems on both sides, pictures seen by twoeyes of the user are the same, and the user does not feel dizzy.

According to a second aspect, this application provides an imagecalibration method, including: obtaining a pupil position offset of auser and a visual acuity of the user, where the pupil position offset ofthe user is an offset of a pupil of the user relative to an opticalaxis; determining, based on the pupil position offset of the user andthe visual acuity of the user, a first distortion correction parametercorresponding to the pupil position offset of the user and the visualacuity of the user; and correcting a position of each pixel on ato-be-displayed image based on the first distortion correctionparameter.

In a possible implementation, the determining, based on the pupilposition offset of the user and the visual acuity of the user, a firstdistortion correction parameter corresponding to the pupil positionoffset of the user and the visual acuity of the user includes:determining the first distortion correction parameter based on the pupilposition offset of the user, the visual acuity of the user, and a secondmapping relationship. The second mapping relationship indicates acorrespondence between a pupil position offset, a visual acuity, and adistortion correction parameter.

In a possible implementation, the first distortion correction parameterincludes a distortion correction parameter corresponding to a redsubpixel, a distortion correction parameter corresponding to a greensubpixel, and a distortion correction parameter corresponding to a bluesubpixel; and the correcting a position of each pixel on ato-be-displayed image based on the first distortion correction parameterincludes: correcting a position of each red subpixel on theto-be-displayed image based on the distortion correction parametercorresponding to the red subpixel; correcting a position of each greensubpixel on the to-be-displayed image based on the distortion correctionparameter corresponding to the green subpixel; and correcting a positionof each blue subpixel on the to-be-displayed image based on thedistortion correction parameter corresponding to the blue subpixel.

In a possible implementation, the obtaining a pupil position offset of auser includes: obtaining an eye image of the user; recognizing a pupilposition from the eye image; and using a distance between the pupilposition and a reference position as the pupil position offset of theuser.

In a possible implementation, the obtaining a visual acuity of the userincludes: obtaining a first position of each optical lens; and searchinga fourth mapping relationship for a first visual acuity corresponding tothe first position, and using the first visual acuity as the visualacuity of the user. The fourth mapping relationship indicates acorrespondence between a position of each optical lens and a visualacuity.

In the image calibration method, because the distortion correctionparameter stored in the second mapping relationship is obtained bysimulating different eye offsets and different visual acuities, aneffect of performing distortion correction based on the distortioncorrection parameter stored in the second mapping relationship isrelatively ideal.

According to a third aspect, this application provides an imagecalibration method, including: obtaining a visual acuity of a user;determining, based on the visual acuity of the user, a first imageheight corresponding to the visual acuity of the user; and adjusting animage height of a to-be-displayed image based on the first image height.

In a possible implementation, the determining, based on the visualacuity of the user, a first image height corresponding to the visualacuity of the user includes: determining the first image height based onthe visual acuity of the user and a third mapping relationship. Thethird mapping relationship indicates a correspondence between a visualacuity and an image height.

In a possible implementation, the adjusting an image height of ato-be-displayed image based on the first image height includes:adjusting the image height of the to-be-displayed image generated by afirst module to be the same as the first image height. The first moduleis configured to render scene data according to an instruction input bythe user or a default playing order, to generate a corresponding image.

In a possible implementation, the adjusting an image height of ato-be-displayed image based on the first image height includes:adjusting a field of view parameter value of a first module based on thefirst image height, so that the image height of the to-be-displayedimage generated by the first module is the same as the first imageheight. The first module is configured to render scene data according toan instruction input by the user or a default playing order, to generatea corresponding image.

In a possible implementation, the obtaining a visual acuity of the userincludes: obtaining a first position of each optical lens; and searchinga fourth mapping relationship for a first visual acuity corresponding tothe first position, and using the first visual acuity as the visualacuity of the user. The fourth mapping relationship indicates acorrespondence between a position of each optical lens and a visualacuity.

In the image calibration method, users with different visual acuitiescan have a same field of view when using a head-mounted display device,thereby enhancing user experience.

According to a fourth aspect, this application provides an electronicdevice, including a camera, a lens position detector, and a processor.The camera is configured to shoot an eye image, the lens positiondetector is configured to detect a position of an optical lens, and theprocessor is configured to perform the method according to the firstaspect, the second aspect, or the third aspect.

According to a fifth aspect, this application provides an electronicdevice, including a memory and a processor. The processor is configuredto be coupled to the memory, and read and execute instructions in thememory, to implement the method according to the first aspect, thesecond aspect, or the third aspect.

According to a sixth aspect, this application provides a readablestorage medium. The readable storage medium stores a computer program,and when the computer program is executed, the method according to thefirst aspect, the second aspect, or the third aspect is implemented.

According to the image calibration method and the device that areprovided in this application, after the pupil position offset of theuser and the visual acuity of the user are detected, a correspondingimage center point offset may be found from the first mappingrelationship based on the pupil position offset of the user and thevisual acuity of the user, and a center point of an image to bedisplayed on the display may be adjusted based on the image center pointoffset. In this way, when the user wears the head-mounted display deviceimproperly, a point seen by the user when the user looks straight aheadis the center point of the image displayed on the display. In addition,after the foregoing calibration is performed on images corresponding tooptical systems on both sides, pictures seen by two eyes of the user arethe same, and the user does not feel dizzy. In addition, a correspondingdistortion correction parameter is found from the second mappingrelationship based on the pupil position offset of the user and thevisual acuity of the user, and based on the distortion correctionparameter, distortion correction is performed on the image to bedisplayed on the display. Because the distortion correction parameterstored in the second mapping relationship is obtained by simulatingdifferent eye offsets and different visual acuities, an effect ofperforming distortion correction based on the distortion correctionparameter stored in the second mapping relationship is relatively ideal.In addition, a corresponding image height is found from the thirdmapping relationship based on the visual acuity of the user, and animage height of the image to be displayed on the display is adjustedbased on the image height. In this way, users with different visualacuities can have a same field of view when using the head-mounteddisplay device, thereby enhancing user experience.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram of an application scenario according to anembodiment of this application;

FIG. 2 is a schematic diagram of an imaging principle according to anembodiment of this application;

FIG. 3 is a schematic diagram of an optical axis according to anembodiment of this application;

FIG. 4 is a framework diagram of a head-mounted display device 100according to an embodiment of this application;

FIG. 5 is a schematic diagram of positions of a camera 404 and a lensposition detector 405 according to an embodiment of this application;

FIG. 6 is a schematic flowchart of an embodiment of an image calibrationmethod according to this application;

FIG. 7A is a schematic diagram of a principle of determining a pupilposition offset of a user according to an embodiment of thisapplication;

FIG. 7B is a schematic diagram of a principle of determining a referenceposition according to an embodiment of this application;

FIG. 8 is a schematic diagram of a pupil position shift according to anembodiment of this application;

FIG. 9 is a schematic diagram of adjusting a center point of ato-be-displayed image according to an embodiment of this application;

FIG. 10 is a schematic diagram of an on-axis distortion according to anembodiment of this application;

FIG. 11 is a schematic diagram of an off-axis distortion according to anembodiment of this application;

FIG. 12 is a schematic diagram of an image distortion degree changingwith a field of view according to an embodiment of this application;

FIG. 13 is a schematic diagram of fields of view corresponding todifferent visual acuities according to an embodiment of thisapplication;

FIG. 14 is a schematic diagram 1 of adjusting an image height of ato-be-displayed image according to an embodiment of this application;

FIG. 15 is a schematic diagram 2 of adjusting an image height of ato-be-displayed image according to an embodiment of this application;and

FIG. 16 is a schematic diagram of a structure of an electronic device10.

DESCRIPTION OF EMBODIMENTS

FIG. 1 is a diagram of an application scenario according to anembodiment of this application. FIG. 1 shows a head-mounted displaydevice 100. The head-mounted display device 100 is a binocular displaydevice. The head-mounted display device 100 includes left and rightoptical systems. An image calibration method provided in embodiments ofthis application is applied to a process in which the two opticalsystems display images. It should be noted that the head-mounted displaydevice 100 shown in FIG. 1 is in a form of glasses, the form is merelyan example, the head-mounted display device 100 may alternatively be inanother form, and the another form includes but is not limited to an eyemask, a helmet, or a head-mounted display. A form of the head-mounteddisplay device 100 is not limited in this embodiment of thisapplication.

In some embodiments, each optical system includes two optical lenses anda display. After a user wears the head-mounted display device 100, theoptical system performs imaging by using a principle shown in FIG. 2 .Refer to FIG. 2 . A pupil 10, an optical lens 101, an optical lens 102,and a display 103 are sequentially arranged from left to right. Afterthe display 103 emits light, the light is refracted by the optical lens102 and the optical lens 101 into the pupil so that the user can see amagnified virtual image. For a range of the virtual image, refer to FIG.2 . This enhances immersion of the user.

It should be noted that FIG. 2 shows an imaging principle of only oneoptical system in the head-mounted display device 100, and an imagingprinciple of the other optical system is similar. The imaging principleof the other optical system is not described in detail in thisembodiment of this application.

Refer to FIG. 3 . For each optical system, an optical axis of theoptical system is formed by a line connecting a center point P1 of theoptical lens 101, a center point P2 of the optical lens 102, and acenter point P3 of the display 103. Still refer to FIG. 2 . When thepupil of the user is on a corresponding optical axis, a point seen bythe user when the user looks straight ahead corresponds to a centerpoint of an image displayed on the display 103, that is, the point seenby the user when the user looks straight ahead is not shifted relativeto the center point of the image displayed on the display 103. When twopupils 10 of the user are both on corresponding optical axes, picturesseen by two eyes are the same, and user experience is high.

However, due to a mismatch of a pupillary distance of the user with adistance between two optical axes of the head-mounted display device100, or improper wearing of the device by the user, or simply otherreasons, the two pupils 10 of the user may be not both on thecorresponding optical axes. As a result, points seen by the user whenthe two eyes of the user look straight ahead are shifted at differentdegrees relative to center points of images displayed on displays 103.Consequently, the two eyes see different pictures, and the user feelsdizzy.

According to the head-mounted display device 100 provided in thisembodiment of this application, in the optical system, the optical lens101 and the optical lens 102 at different positions correspond todifferent visual acuities. The visual acuity mentioned in thisembodiment of this application includes but is not limited to a myopiadegree or a hyperopia degree. Therefore, the user may adjust positionsof the optical lens 101 and the optical lens 102 so that the positionsof the two lenses match a visual acuity of the user. For example, if amyopia degree of a right eye of the user is 700 degrees, when thehead-mounted display device 100 is used, the optical lens 101 and theoptical lens 102 in the right optical system may be adjusted topositions corresponding to myopia of 700 degrees.

It should be noted that, if the pupil 10 of the user is not on thecorresponding optical axis, when the optical lens 101 and the opticallens 102 are at different positions, a point seen by the user when theuser looks straight ahead deviates at different degrees relative to thecenter point of the image displayed on the display 103. That is, whenthe pupil 10 of the user is not on the corresponding optical axis, adegree at which the point seen by the user when the user looks straightahead deviates relative to the center point of the image displayed onthe display 103 is related to the visual acuity of the user.

To resolve the foregoing problem that a user feels dizzy, thisapplication provides an image calibration method. The method may be usedin a process in which two optical systems display images. A pupilposition offset of the user is detected by using a camera, and a visualacuity of the user is detected by using a lens position detector. Then,a corresponding image center point offset is found from a preconfiguredmapping relationship based on the pupil position offset of the user andthe visual acuity of the user. Finally, a center point of an image to bedisplayed on a display is adjusted based on the found image center pointoffset. In this way, a point seen by the user when the user looksstraight ahead is not shifted relative to the center point of the imagedisplayed on the display 103, and after the foregoing calibration isperformed for the two optical systems, pictures seen by two eyes of theuser are the same, and the user does not feel dizzy.

FIG. 4 is a framework diagram of a head-mounted display device 100according to an embodiment of this application. The head-mounted displaydevice 100 includes but is not limited to two optical systems 400, aprocessor 401, and a memory 402. Each optical system 400 includes but isnot limited to a display 103, an optical lens barrel module 403, acamera 404, a lens position detector 405, and a visual acuity adjustmentmodule 406. The optical lens barrel module 403, the display 103, thecamera 404, and the lens position detector 405 in each optical system400 are all connected to the processor 401. The memory 402 is alsoconnected to the processor 401. The memory 402 stores a software unitrelated to image processing, and the software unit includes a virtualcamera. The following describes functions of hardware or software unitsone by one.

In a possible implementation, the optical lens barrel module 403includes an optical lens 101 and an optical lens 102 that are shown inFIG. 2 . The visual acuity adjustment module 406 is configured to adjustpositions of the optical lens 101 and the optical lens 102, so that thepositions of the optical lens 101 and the optical lens 102 match avisual acuity of a user. It should be noted that the optical lens barrelmodule 403 may alternatively include one or more than two lenses. Inthis embodiment of this application, two lenses are used as an exampleto describe a calibration process in this application.

In a possible implementation, the camera 404 is configured to shoot aneye image, and send the eye image to the processor 401.

In a possible implementation, the lens position detector 405 isconfigured to detect a position of each optical lens in the optical lensbarrel module 403, and send the position of each optical lens to theprocessor 401. The position, detected by the lens position detector 405,of each optical lens may be a position of each optical lens relative tothe display 103.

In a possible implementation, positions at which the cameras 404 and thelens position detectors 405 are disposed in the two optical systems 400may be shown in FIG. 5 .

In a possible implementation, the processor 401 is configured todetermine a pupil position offset of the user based on the eye imagesent by the camera 404. The processor is further configured to determinethe visual acuity of the user based on the position, sent by the lensposition detector 405, of each optical lens.

In a possible implementation, the memory 402 further stores anapplication (APP). The APP may be a game APP, or may be a video APP. Atype of the APP is not limited in this application. The game APP is usedas an example. The game APP includes several pre-established gamescenes. Each game scene has corresponding scene data, and acorresponding image may be generated by rendering the scene data. Afterthe user opens the APP, the virtual camera renders the scene dataaccording to an instruction input by the user, to generate acorresponding image. Alternatively, the scene data is rendered in adefault playing order, to generate a corresponding image. The virtualcamera may send a generated to-be-displayed image to the display 103,and the display 103 is configured to display the image sent by thevirtual camera.

In a possible implementation, the virtual camera includes a displacementparameter and a field of view parameter. A center point of the imagegenerated by the virtual camera may be adjusted by changing adisplacement parameter value of the virtual camera, and an image heightof the image generated by the virtual camera may be adjusted by changinga field of view parameter value of the virtual camera.

In a possible implementation, the processor 401 is further configured toadjust, based on the pupil position offset of the user and the visualacuity of the user, a center point of an image to be displayed on thedisplay 103. The processor is further configured to determine, based onthe pupil position offset of the user and the visual acuity of the user,a distortion correction parameter corresponding to the image to bedisplayed on the display 103, and perform, based on the distortioncorrection parameter, distortion correction on the image to be displayedon the display 103. The processor is further configured to adjust, basedon the visual acuity of the user, an image height of the image to bedisplayed on the display 103. After the foregoing image calibration, avirtual image seen by the user is not affected by a pupil positionshift, nor affected by the visual acuity. After the foregoingcalibration is performed on the two optical systems, pictures seen bytwo eyes of the user are the same, which improves user experience ofusing the head-mounted display device 100.

In a possible implementation, the head-mounted display device 100further includes a pupillary distance based mechanical adjustmentapparatus. The pupillary distance based mechanical adjustment apparatusis configured to adjust a distance between two optical axes, so that thedistance between the two optical axes matches a pupillary distance ofthe user. However, after the adjustment of the pupillary distance basedmechanical adjustment device, if the adjustment is inappropriate orpupils of the user are shifted upward or downward relative to opticalaxes after the user wears the head-mounted display device 100, the twopupils 10 of the user are still not on corresponding optical axes. Inthis case, an image to be displayed on the display 103 may be calibratedby using the image calibration method provided in embodiments of thisapplication.

FIG. 6 is a schematic flowchart of an embodiment of an image calibrationmethod according to this application, applied to the head-mounteddisplay device 100 shown in FIG. 4 . The image calibration methodprovided in this embodiment may be applied after the user triggers andstarts the head-mounted display device 100, or may be applied after theuser triggers and starts an APP installed in the head-mounted displaydevice 100. This is not limited in this embodiment of this application.The image calibration method provided in this embodiment specificallyincludes the following steps.

S601. The camera 404 shoots an eye image, and sends the eye image to theprocessor 401.

S602. The lens position detector 405 detects a first position of eachoptical lens, and sends the first position of each optical lens to theprocessor 401.

S603. The processor 401 determines a pupil position offset of a userbased on the eye image.

In a possible implementation, refer to FIG. 7A. After the eye image sentby the camera 404 is received, a pupil position is recognized from theeye image, and then a distance between the pupil position and areference position is calculated as the pupil position offset of theuser.

The following describes how to obtain the reference position.

Refer to FIG. 7B. A positioning point is disposed on the optical lensbarrel module 403, and a notch matching the positioning point isdisposed on tooling. After a calibration object is installed on thetooling, the positioning point is clamped in the notch, to ensure thatthe calibration object is on an optical axis. The camera 404 is used toshoot an image of the calibration object, a center point of thecalibration object is recognized from the image of the calibrationobject, and a position of the center point is used as the referenceposition.

S604. The processor 401 determines a visual acuity of the user based onthe first position of each optical lens.

In a possible implementation, after receiving the first position, sentby the lens position detector 405, of each optical lens, the processor401 may find, from a fourth mapping relationship, a first visual acuitycorresponding to the first position, and use the found first visualacuity as the visual acuity of the user. The fourth mapping relationshipis a preconfigured correspondence between a position of each opticallens and a visual acuity.

In a possible implementation, in the fourth mapping relationship, thevisual acuity may include 0 degree, 100 degrees, 200 degrees, 300degrees, 400 degrees, 500 degrees, 600 degrees, and 700 degrees, and aposition, corresponding to each visual acuity, of each optical lens isshown in Table 1. A position in Table 1 is a position of the opticallens relative to the display 103. It should be noted that thecorrespondence shown in Table 1 is merely an example. The correspondencebetween a position of the optical lens 101, a position of the opticallens 102, and a visual acuity may alternatively be another relationship.Table 1 constitutes no limitation on this application.

TABLE 1 Optical lens 101 Optical lens 102 Visual acuity Position A1Position B1  0 degree Position A2 Position B2 100 degrees Position A3Position B3 200 degrees Position A4 Position B4 300 degrees Position A5Position B5 400 degrees Position A6 Position B6 500 degrees Position A7Position B7 600 degrees Position A8 Position B8 700 degrees

To enable more users with visual acuities to use the head-mounteddisplay device 100, the visual acuity may be designed as a function ofpositions of the two optical lenses. After wearing the head-mounteddisplay device 100, the user adjusts the positions of the two opticallenses, so that a visual acuity calculated by using the function matchesthe visual acuity of the user. In this design, the head-mounted displaydevice 100 can be used by a user with any visual acuity.

The following uses an example for description with reference to Table 1.

Assuming that the lens position detector 405 detects that the opticallens 101 is at the position A3 and the optical lens 102 is at theposition B3, the processor 401 may find, from the fourth mappingrelationship, that the visual acuity of the user is 200 degrees. S605.The processor 401 adjusts, based on the pupil position offset of theuser and the visual acuity of the user, a center point of an image to bedisplayed on the display 103.

In a possible implementation, the processor 401 finds a correspondingfirst image center point offset from a first mapping relationship basedon the pupil position offset of the user obtained in S603 and the visualacuity of the user obtained in S604, and adjust, based on the firstimage center point offset and a pupil shift direction, the center pointof the image to be displayed on the display. The first mappingrelationship is a preconfigured correspondence between a pupil positionoffset, a visual acuity, and an image center point offset. The image tobe displayed on the display is also referred to as a to-be-displayedimage in this application.

In a possible implementation, in the first mapping relationship, thevisual acuity may include 0 degree, 100 degrees, 200 degrees, 300degrees, 400 degrees, 500 degrees, 600 degrees, and 700 degrees, and thepupil position offset may include 1 mm, 2 mm, 3 mm, and 4 mm. An imagecenter point offset corresponding to each visual acuity and each pupilposition offset is shown in Table 2. It should be noted that thecorrespondence shown in Table 2 is merely an example. The correspondencebetween a pupil position offset, a visual acuity, and an image centerpoint offset may alternatively be another relationship. Table 2constitutes no limitation on this application.

TABLE 2 Pupil position Image center Visual acuity offset point offset  0degree 1 mm A1 mm 2 mm B1 mm 3 mm C1 mm 4 mm D1 mm 100 degrees 1 mm A2mm 2 mm B2 mm 3 mm C2 mm 4 mm D2 mm 200 degrees 1 mm A3 mm 2 mm B3 mm 3mm C3 mm 4 mm D3 mm 300 degrees 1 mm A3 mm 2 mm B3 mm 3 mm C3 mm 4 mm D3mm 400 degrees 1 mm A4 mm 2 mm B4 mm 3 mm C4 mm 4 mm D4 mm 500 degrees 1mm A5 mm 2 mm B5 mm 3 mm C5 mm 4 mm D5 mm 600 degrees 1 mm A6 mm 2 mm B6mm 3 mm C6 mm 4 mm D6 mm 700 degrees 1 mm A7 mm 2 mm B7 mm 3 mm C7 mm 4mm D7 mm

The following uses an example in which the pupil position offset is 2 mmand the visual acuity is 200 degrees to describe a process ofconfiguring the first mapping relationship.

Refer to FIG. 8 . A simulation manner may be used to simulate that thepupil is shifted 2 mm upward relative to the optical axis, and based onthe first mapping relationship shown in Table 1, the optical lens 101and the optical lens 102 may be adjusted to positions corresponding to200 degrees. In this case, a point Q1 seen by the user when the userlooks straight ahead is not on the optical axis, a direction in which Q1deviates from the optical axis is the same as a direction in which thepupil deviates from the optical axis, and a distance by which Q1deviates from the optical axis is the same as a distance by which thepupil deviates from the optical axis. That is, Q1 is also shifted 2 mmupwards relative to the optical axis. Q1 does not correspond to thecenter point of the image displayed on the display 103, but a pointobtained when the center point of the image displayed on the display 103is shifted upward by a distance, which is represented by P4 in FIG. 8 .Only when the center point of the image displayed on the display 103 ismoved upward from P3 to P4, a point seen by the user when the user looksstraight ahead is the center point of the image displayed on the display103. Therefore, a distance A3 mm from P3 to P4 may be used as an imagecenter point offset corresponding to the pupil position offset of 2 mmand the visual acuity of 200 degrees.

In a possible implementation, the processor 401 finds a correspondingfirst image center point offset from the first mapping relationshipbased on the pupil position offset of the user obtained in S603 and thevisual acuity of the user obtained in S604, and then may move a centerpoint of a to-be-displayed image generated by the virtual camera towarda pupil shift direction by a first distance. The first distance is thesame as the first image center point offset.

For example:

It is assumed that the pupil position offset of the user obtained inS603 is 2 mm, and the visual acuity of the user obtained in S604 is 200degrees. It may be found from the first mapping relationship that acorresponding image center point offset is A3 mm. Refer to FIG. 9 . Theprocessor 401 may move the center point of the to-be-displayed imagegenerated by the virtual camera toward the pupil shift direction by A3mm, so that a point seen by the user when the user looks straight aheadis the center point of the image displayed on the display. After theforegoing calibration is performed on the optical systems on both sides,pictures seen by two eyes of the user are the same, and the user doesnot feel dizzy.

It should be noted that an adjustment process in this application isdescribed in FIG. 9 by using an example in which the pupil shiftdirection is an upward shift. An adjustment process when the pupil isshifted in another direction is similar, and details are not describedin this embodiment of this application.

In another possible implementation, the processor 401 finds acorresponding first image center point offset from the first mappingrelationship based on the pupil position offset of the user obtained inS603 and the visual acuity of the user obtained in S604, and then mayperform conversion based on the first image center point offset toobtain a displacement adjustment amount of the virtual camera, andadjust a displacement parameter value of the virtual camera based on thedisplacement adjustment amount, so that a center point of ato-be-displayed image generated by the virtual camera is moved toward apupil shift direction by a first distance. The first distance is thesame as the first image center point offset.

For example:

It is assumed that the pupil position offset of the user obtained inS603 is 2 mm, and the visual acuity of the user obtained in S604 is 200degrees. It may be found from the first mapping relationship that acorresponding image center point offset is A3 mm. The displacementadjustment amount, obtained through conversion, of the virtual camera isa3 mm. The displacement parameter value of the virtual camera may beadjusted based on the displacement adjustment amount. After theadjustment, the center point of the image generated by the virtualcamera is moved A3 mm toward the pupil shift direction, so that a pointseen by the user when the user looks straight ahead is the center pointof the image displayed on the display. After the foregoing calibrationis performed on the optical systems on both sides, pictures seen by twoeyes of the user are the same, and the user does not feel dizzy.

S606. The processor 401 performs, based on the pupil position offset ofthe user and the visual acuity of the user, distortion correction on theimage to be displayed on the display.

In a possible implementation, the image to be displayed on the displayin S606 may be an image obtained after the processor 401 performs S605.

Because the optical lens 101 and the optical lens 102 have differentmagnifications at an edge part and a central part, an image seen by theuser is distorted. FIG. 10 and FIG. 11 are schematic diagrams of twodistortions of a same image that are seen by a user. FIG. 10 is aschematic diagram of an on-axis distortion. FIG. 11 is a schematicdiagram of an off-axis distortion. The on-axis distortion refers to adistortion seen by the user when the pupil is not shifted relative tothe optical axis. The off-axis distortion refers to a distortion seen bythe user when the pupil is shifted relative to the optical axis. FIG. 11illustrates a distortion seen when the pupil is shifted upward relativeto the optical axis. In FIG. 10 and FIG. 11 , a grid intersection pointrepresents a position of a pixel when no distortion occurs, and across-shaped point represents a position of a pixel after an image isdistorted. It can be learned by comparison that an image distortionstatus seen by the user when the pupil is not shifted is different froman image distortion status seen by the user when the pupil is shifted.For example, when the pupil is not shifted, a pixel at P0 is moved to P1in FIG. 10 after a distortion, as shown by an arrow in FIG. 10 . Whenthe pupil is shifted, the pixel at P0 is moved to P2 in FIG. 11 after adistortion, as shown by an arrow in FIG. 11 . A position shown by P1 inFIG. 10 is obviously different from a position shown by P2 in FIG. 11 .

In addition, refer to FIG. 12 . In upper and lower figures, a horizontalcoordinate represents a distortion degree, and a vertical coordinaterepresents a field of view. The upper figure shows an image distortiondegree changing with a field of view when a visual acuity is 700degrees. The lower figure shows an image distortion degree changing witha field of view when a visual acuity is 0 degree. It can be learned bycomparison that users with different visual acuities see different imagedistortion degrees, that is, the image distortion degree is related tothe visual acuity.

In a possible implementation, the processor 401 finds a correspondingfirst distortion correction parameter from a second mapping relationshipbased on the pupil position offset of the user obtained in S603 and thevisual acuity of the user obtained in S604. For each pixel of theto-be-displayed image, the first distortion correction parameter andcoordinates of the pixel may be substituted into a distortion correctionformula, to obtain coordinates of the pixel after distortion correction.The second mapping relationship is a preconfigured correspondencebetween a pupil position offset, a visual acuity, and a distortioncorrection parameter.

For example:

It is assumed that the distortion correction formula isR′=a*R+b*R³+c*R⁵+d*R⁷+e*R⁹, where a, b, c, d, and e represent the firstdistortion correction parameter found from the second mappingrelationship, R represents a distance between a pixel before distortioncorrection and the center point of the image, R may be calculated basedon coordinates of the pixel before distortion correction and coordinatesof the center point of the image, and R′ represents a distance between apixel after distortion correction and the center point of the image. Foreach pixel on the to-be-displayed image, R′ is obtained, and the pixelis moved along a connection line between the pixel before distortioncorrection and the center point of the image to a position correspondingto R′, to complete distortion correction of the pixel.

In a possible implementation, the distortion correction formula mayalternatively be a polynomial formula used in the following polynomialfitting process.

In a possible implementation, in the second mapping relationship, thevisual acuity may include 0 degree, 100 degrees, 200 degrees, 300degrees, 400 degrees, 500 degrees, 600 degrees, and 700 degrees, and thepupil position offset may include 1 mm, 2 mm, 3 mm, and 4 mm. For adistortion correction parameter corresponding to each visual acuity andeach pupil position offset, refer to Table 3. It should be noted thatthe correspondence shown in Table 3 is merely an example. Thecorrespondence between a pupil position offset, a visual acuity, and adistortion correction parameter may alternatively be anotherrelationship. In addition, the distortion correction parameter mayalternatively include a plurality of values. Table 3 constitutes nolimitation on this application.

TABLE 3 Distortion correction Visual acuity Eye offset parameter  0degree 1 mm A1 2 mm B1 3 mm C1 4 mm D1 100 degrees 1 mm A2 2 mm B2 3 mmC2 4 mm D2 200 degrees 1 mm A3 2 mm B3 3 mm C3 4 mm D3 300 degrees 1 mmA4 2 mm B4 3 mm C4 4 mm D4 400 degrees 1 mm A5 2 mm B5 3 mm C5 4 mm D5500 degrees 1 mm A6 2 mm B6 3 mm C6 4 mm D6 600 degrees 1 mm A7 2 mm B73 mm C7 4 mm D7 700 degrees 1 mm A8 2 mm B8 3 mm C8 4 mm D8

The following uses an example in which the pupil position offset is 2 mmand the visual acuity is 200 degrees to describe a process ofconfiguring the second mapping relationship.

A simulation manner may be used to simulate that the eye position isshifted 2 mm upward relative to the optical axis, and based on the firstmapping relationship shown in Table 1, the optical lens 101 and theoptical lens 102 may be adjusted to positions corresponding to 200degrees. In this case, it is assumed that an image distortion statusseen by the user is shown in FIG. 11 . A position of each pixel when nodistortion occurs and a position of each pixel after the image isdistorted are obtained from FIG. 11 . Then, the two are fit in apolynomial fitting manner. A fitting result is a distortion correctionparameter corresponding to the image distortion status shown in FIG. 11. The distortion correction parameter may be used as a distortioncorrection parameter corresponding to the pupil position offset of 2 mmand the visual acuity of 200 degrees, and stored in the second mappingrelationship. Because the distortion correction parameter stored in thesecond mapping relationship is obtained by simulating different eyeoffsets and different visual acuities, an effect of performingdistortion correction based on the distortion correction parameterstored in the second mapping relationship is relatively ideal.

Each pixel on the image includes red, blue, and green subpixels. Afterthe image is distorted, the three subpixels are moved to differentpositions, resulting in color separation of the image displayed on thedisplay. Therefore, the foregoing distortion correction may beseparately performed on the red subpixel, the blue subpixel, and thegreen subpixel, to resolve the color separation problem.

In a possible implementation, when the second mapping relationship isconfigured, a correspondence between a pupil position offset, a visualacuity, and a distortion correction parameter corresponding to asubpixel of each color may be configured. In the second mappingrelationship, the visual acuity may include 0 degree, 100 degrees, 200degrees, 300 degrees, 400 degrees, 500 degrees, 600 degrees, and 700degrees, the pupil position offset may include 1 mm, 2 mm, 3 mm, and 4mm, and the distortion correction parameter may include a distortioncorrection parameter corresponding to the red subpixel, a distortioncorrection parameter corresponding to the blue subpixel, and adistortion correction parameter corresponding to the green subpixel, asshown in Table 4. The correspondence shown in Table 4 is merely anexample. The correspondence between a pupil position offset, a visualacuity, and a distortion correction parameter of each color mayalternatively be another relationship. Table 4 constitutes no limitationon this application.

TABLE 4 Distortion Distortion Distortion correction correctioncorrection parameter parameter parameter corresponding correspondingcorresponding Visual Eye to red to blue to green acuity offset subpixelsubpixel subpixel  0 degree 1 mm A1 E1 M1 2 mm B1 F1 N1 3 mm C1 G1 P1 4mm D1 H1 Q1 100 degrees 1 mm A2 E2 M2 2 mm B2 F2 N2 3 mm C2 G2 P2 4 mmD2 H2 Q2 200 degrees 1 mm A3 E3 M3 2 mm B3 F3 N3 3 mm C3 G3 P3 4 mm D3H3 Q3 300 degrees 1 mm A4 E4 M4 2 mm B4 F4 N4 3 mm C4 G4 P4 4 mm D4 H4Q4 400 degrees 1 mm A5 E5 M5 2 mm B5 F5 N5 3 mm C5 G5 P5 4 mm D5 H5 Q5500 degrees 1 mm A6 E6 M6 2 mm B6 F6 N6 3 mm C6 G6 P6 4 mm D6 H6 Q6 600degrees 1 mm A7 E7 M7 2 mm B7 F7 N7 3 mm C7 G7 P7 4 mm D7 H7 Q7 700degrees 1 mm A8 E8 M8 2 mm B8 F8 N8 3 mm C8 G8 P8 4 mm D8 H8 Q8

In a possible implementation, the processor 401 finds, based on thepupil position offset of the user obtained in S603 and the visual acuityof the user obtained in S604, a corresponding distortion correctionparameter corresponding to the red subpixel, a corresponding distortioncorrection parameter corresponding to the blue subpixel, and acorresponding distortion correction parameter corresponding to the greensubpixel from the second mapping relationship shown in Table 4. For eachred subpixel on the to-be-displayed image, the processor substitutes thedistortion correction parameter corresponding to the red subpixel andcoordinates of the red subpixel into the distortion correction formula,to obtain coordinates of the red subpixel after distortion correction.For each green subpixel on the to-be-displayed image, the processorsubstitutes the distortion correction parameter corresponding to thegreen subpixel and coordinates of the green subpixel into the distortioncorrection formula, to obtain coordinates of the green subpixel afterdistortion correction. For each blue subpixel on the to-be-displayedimage, the processor substitutes the distortion correction parametercorresponding to the blue subpixel and coordinates of the blue subpixelinto the distortion correction formula, to obtain coordinates of theblue subpixel after distortion correction. For a process of performingdistortion correction on the subpixel of each color, refer to theforegoing description.

S607. The processor 401 adjusts, based on the visual acuity of the user,an image height of the image to be displayed on the display.

Refer to FIG. 13 . For example, the optical lens barrel module 403includes only one optical lens 101. When the optical lens 101 is at asolid line position, a corresponding field of view (FOV) is α. When theoptical lens 101 is moved to a dotted line position, a correspondingfield of view is β. Because a position of the optical lens changes avisual acuity, a field of view of the user is related to the visualacuity of the user. To enable users with different visual acuities tohave a same field of view when using the head-mounted display device100, the processor 401 may adjust, based on the visual acuities of theusers, the image height of the image to be displayed on the display.

In a possible implementation, the processor 401 finds a correspondingfirst image height from a third mapping relationship based on the visualacuity of the user obtained in S604, and adjusts, based on the firstimage height, the image height of the image to be displayed on thedisplay. The third mapping relationship is a preconfiguredcorrespondence between a visual acuity and an image height.

In a possible implementation, in the third mapping relationship, thevisual acuity may include 0 degree, 100 degrees, 200 degrees, 300degrees, 400 degrees, 500 degrees, 600 degrees, and 700 degrees, and animage height corresponding to each visual acuity is shown in Table 5. Itshould be noted that the correspondence shown in Table 5 is merely anexample. The correspondence between a visual acuity and an image heightmay alternatively be another relationship. Table 5 constitutes nolimitation on this application.

TABLE 5 Visual acuity Image height  0 degree H1 mm 100 degrees H2 mm 200degrees H3 mm 300 degrees H4 mm 400 degrees H5 mm 500 degrees H6 mm 600degrees H7 mm 700 degrees H8 mm

The following uses an example in which the visual acuity is 200 degreesto describe a process of configuring the third mapping relationship.

In FIG. 13 , it is assumed that a corresponding visual acuity when theoptical lens 101 is at the solid line position is 0 degree, acorresponding visual acuity when the optical lens 101 is moved to thedotted line position is 200 degrees, a field of view of a user whosevisual acuity is 0 degree is α, and a field of view of a user whosevisual acuity is 200 degrees is β. To enable the user whose visualacuity is 200 degrees and the user whose visual acuity is 0 degree tohave a same field of view, when the third mapping relationship isconfigured, refer to FIG. 14 . The image height of the image to bedisplayed on the display 103 may be adjusted until the field of view ofthe user whose visual acuity is 200 degrees changes to a. In this case,the image height is H3 mm in Table 5.

In a possible implementation, the processor 401 finds a correspondingfirst image height from a third mapping relationship based on the visualacuity of the user obtained in S604, and then may adjust an image heightof a to-be-displayed image generated by the virtual camera to be thesame as the first image height. Because the image height stored in thethird mapping relationship is obtained by simulating different visualacuities, after the image to be displayed on the display 103 is adjustedbased on the found image height, a field of view of the user is the sameas a field of view of a user with another visual acuity.

For example:

It is assumed that the visual acuity of the user obtained in S604 is 200degrees. It may be found from the third mapping relationship that acorresponding image height is H3 mm. Refer to FIG. 15 . The processor401 may adjust an image height of an image shot by the virtual camera toH3 mm. When the image height is adjusted to H3 mm, fields of view ofboth a user whose visual acuity is 200 degrees and a user whose visualacuity is 0 degree when using the head-mounted display device 100 are α,thereby enhancing user experience.

In another possible implementation, the processor 401 finds acorresponding first image height from a third mapping relationship basedon the visual acuity of the user obtained in S604, and then may performconversion based on the image height to obtain a field of viewadjustment amount of the virtual camera, and adjust a field of viewparameter value of the virtual camera based on the field of viewadjustment amount, so that an image height of a to-be-displayed imagegenerated by the virtual camera is the same as the first image height.

For example:

It is assumed that the visual acuity of the user obtained in S604 is 200degrees. In this case, it may be found from the third mappingrelationship that a corresponding image height is H3 mm, and the fieldof view adjustment amount, obtained through conversion, of the virtualcamera is x degrees. The field of view parameter value of the virtualcamera may be adjusted based on the field of view adjustment amount.After the adjustment, the image height of the image generated by thevirtual camera becomes H3 mm. Similarly, fields of view of both a userwhose visual acuity is 200 degrees and a user whose visual acuity is 0degree when using the head-mounted display device 100 are α, therebyenhancing user experience.

It should be noted that a sequence of S601 and S602 is not limited inthis embodiment of this application. S601 may be performed before S602,or S602 may be performed before S601, or S601 and S602 may be performedat the same time. Similarly, a sequence of S603 and S604 is not limitedin this embodiment of this application. S603 may be performed beforeS604, or S604 may be performed before S603, or S603 and S604 areperformed at the same time. Similarly, a sequence of S605, S606, andS607 is not limited in this embodiment of this application. S605, S606,and S607 may be performed in any sequence. A sequence in FIG. 6 ismerely an example.

According to the image calibration method provided in this embodiment ofthis application, after the pupil position offset of the user and thevisual acuity of the user are detected, a corresponding image centerpoint offset may be found from the first mapping relationship based onthe pupil position offset of the user and the visual acuity of the user,and a center point of an image to be displayed on the display may beadjusted based on the image center point offset. In this way, a pointseen by the user when the user looks straight ahead is the center pointof the image displayed on the display, and after the foregoingcalibration is performed on the optical systems on both sides, picturesseen by the two eyes of the user are the same, and the user does notfeel dizzy. In addition, a corresponding distortion correction parameteris found from the second mapping relationship based on the pupilposition offset of the user and the visual acuity of the user, and basedon the distortion correction parameter, distortion correction isperformed on the image to be displayed on the display. Because thedistortion correction parameter stored in the second mappingrelationship is obtained by simulating different eye offsets anddifferent visual acuities, an effect of performing distortion correctionbased on the distortion correction parameter stored in the secondmapping relationship is relatively ideal. In addition, a correspondingimage height is found from the fourth mapping relationship based on thevisual acuity of the user, and an image height of the image to bedisplayed on the display is adjusted based on the image height. In thisway, users with different visual acuities can have a same field of viewwhen using the head-mounted display device, thereby enhancing userexperience.

FIG. 16 is a schematic diagram of a structure of an electronic device10. The electronic device 10 may include a processor 110, an externalmemory interface 120, an internal memory 121, a universal serial bus(USB) interface 130, an antenna 1, an antenna 2, a mobile communicationsmodule 150, a wireless communications module 160, an audio module 170, aspeaker 170A, a receiver 170B, a microphone 170C, a headset interface170D, a sensor module 180, a button 190, a motor 191, an indicator 192,a camera 193, a display 194, and the like. The sensor module 180 mayinclude a pressure sensor 180A, a gyroscope sensor 180B, an accelerationsensor 180E, a distance sensor 180F, a touch sensor 180K, and the like.

It may be understood that the structure shown in this embodiment of thepresent disclosure does not constitute a specific limitation on theelectronic device 10. In some other embodiments of this application, theelectronic device 10 may include more or fewer components than thoseshown in the figure, or have some components combined, or have somecomponents split, or have a different component arrangement. Thecomponents shown in the figure may be implemented by hardware, software,or a combination of software and hardware.

The processor 110 may include one or more processing units. For example,the processor 110 may include an application processor (AP), a modemprocessor, a graphics processing unit (GPU), an image signal processor(ISP), a controller, a video codec, a digital signal processor (DSP), abaseband processor, a neural network processing unit (NPU), and/or thelike. Different processing units may be independent components, or maybe integrated into one or more processors.

The controller may generate an operation control signal based oninstruction operation code and a timing signal, to control instructionfetching and instruction execution.

A memory may be further disposed in the processor 110, to storeinstructions and data. In some embodiments, the memory in the processor110 is a cache. The memory may store instructions or data that has beenused or is cyclically used by the processor 110. If the processor 110needs to use the instructions or the data again, the processor maydirectly invoke the instructions or the data from the memory. Thisavoids repeated access, and reduces a waiting time of the processor 110,thereby improving system efficiency.

The processor 110 is configured to adjust, based on a pupil positionoffset of a user and a visual acuity of the user, a center point of animage to be displayed on the display 194. The processor is furtherconfigured to determine, based on the pupil position offset of the userand the visual acuity of the user, a distortion correction parametercorresponding to the image to be displayed on the display 194, andperform, based on the distortion correction parameter, distortioncorrection on the image to be displayed on the display 194. Theprocessor is further configured to adjust, based on the visual acuity ofthe user, an image height of the image to be displayed on the display194. After the foregoing image calibration, a virtual image seen by theuser is not affected by an eye position shift, nor affected by an eyevisual acuity. After the foregoing calibration is performed on twooptical systems, pictures seen by two eyes of the user are the same,which improves user experience of using the electronic device 10.

In some embodiments, the processor 110 may include one or moreinterfaces. The interface may include an inter-integrated circuit (I2C)interface, an inter-integrated circuit sound (I2S) interface, a pulsecode modulation (PCM) interface, a universal asynchronousreceiver/transmitter (UART) interface, a mobile industry processorinterface (MIPI), a general-purpose input/output (GPIO) interface, asubscriber identity module (SIM) interface, a universal serial bus (USB)interface, and/or the like.

The I2C interface is a bidirectional synchronous serial bus, andincludes a serial data line (SDA) and a serial clock line (SCL). In someembodiments, the processor 110 may include a plurality of groups of I2Cbuses. The processor 110 may be respectively coupled to the touch sensor180K, a charger, a flash, a camera 193, and the like through differentI2C bus interfaces. For example, the processor 110 may be coupled to thetouch sensor 180K through an I2C interface, so that the processor 110communicates with the touch sensor 180K through the I2C bus interface,to implement a touch function of the electronic device 10.

The I2S interface may be configured for audio communication. In someembodiments, the processor 110 may include a plurality of groups of I2Sbuses. The processor 110 may be coupled to the audio module 170 by usingan I2S bus, to implement communication between the processor 110 and theaudio module 170. In some embodiments, the audio module 170 may transferan audio signal to the wireless communications module 160 through an I2Sinterface, to implement a function of answering a call by using aBluetooth headset.

The PCM interface may also be configured for audio communication, tosample, quantize, and encode an analog signal. In some embodiments, theaudio module 170 and the wireless communications module 160 may becoupled through a PCM bus interface. In some embodiments, the audiomodule 170 may alternatively transfer an audio signal to the wirelesscommunications module 160 through the PCM interface, to implement afunction of answering a call by using a Bluetooth headset. Both the I2Sinterface and the PCM interfaces may be configured for audiocommunication.

The UART interface is a universal serial data bus, and is configured forasynchronous communication. The bus may be a bidirectionalcommunications bus. It converts to-be-transmitted data between serialcommunication and parallel communication. In some embodiments, the UARTinterface is usually configured to connect the processor 110 to thewireless communications module 160. For example, the processor 110communicates with a Bluetooth module in the wireless communicationsmodule 160 through the UART interface, to implement a Bluetoothfunction. In some embodiments, the audio module 170 may transfer anaudio signal to the wireless communications module 160 through the UARTinterface, to implement a function of playing music by using a Bluetoothheadset.

The MIPI interface may be configured to connect the processor 110 to aperipheral device like the display 194 and the camera 193. The MIPIinterface includes a camera serial interface (CSI), a display serialinterface (DSI), and the like. In some embodiments, the processor 110communicates with the camera 193 through the CSI interface, to implementa photographing function of the electronic device 10. The processor 110communicates with the display 194 through the DSI interface, toimplement a display function of the electronic device 10.

The GPIO interface may be configured by using software. The GPIOinterface may be configured for a control signal or a data signal. Insome embodiments, the GPIO interface may be configured to connect theprocessor 110 to the camera 193, the display 194, the wirelesscommunications module 160, the audio module 170, the sensor module 180,and the like. The GPIO interface may be further configured as an I2Cinterface, an I2S interface, a UART interface, an MIPI interface, or thelike.

The USB interface 130 is an interface that complies with a USB standardspecification, and may be specifically a Mini USB interface, a Micro USBinterface, a USB Type C interface, or the like. The USB interface 130may be configured to connect to a charger to charge the electronicdevice 10, and may be configured to transmit data between the electronicdevice 10 and a peripheral device. The USB interface may also beconfigured to connect to a headset, to play audio by using the headset.The interface may be further configured to connect to another electronicdevice, for example, an AR device.

It may be understood that the schematic interfacing relationship betweenmodules in this embodiment of the present disclosure is merely anexample for description, and does not constitute a limitation on astructure of the electronic device 10. In some other embodiments of thisapplication, the electronic device 10 may alternatively use aninterfacing manner different from that in the foregoing embodiment, oruse a combination of a plurality of interfacing manners.

A wireless communications function of the electronic device 10 may beimplemented by using the antenna 1, the antenna 2, the mobilecommunications module 150, the wireless communications module 160, themodem processor, the baseband processor, and the like.

The antenna 1 and the antenna 2 are configured to transmit and receiveelectromagnetic wave signals. Each antenna in the electronic device 10may be configured to cover one or more communications bands. Differentantennas may be multiplexed to improve antenna utilization. For example,the antenna 1 may be multiplexed as a diversity antenna of a wirelesslocal area network. In some other embodiments, the antenna may be usedin combination with a tuning switch.

The mobile communications module 150 may provide a wirelesscommunications solution that is applied to the electronic device 10 andthat includes 2G/3G/4G/5G. The mobile communications module 150 mayinclude at least one filter, switch, power amplifier, low noiseamplifier (LNA), and the like. The mobile communications module 150 mayreceive an electromagnetic wave by using the antenna 1, performprocessing such as filtering or amplification on the receivedelectromagnetic wave, and transmit the electromagnetic wave to the modemprocessor for demodulation. The mobile communications module 150 mayfurther amplify a signal modulated by the modem processor, and convertthe signal into an electromagnetic wave for radiation by using theantenna 1. In some embodiments, at least some functional modules of themobile communications module 150 may be disposed in the processor 110.In some embodiments, at least some functional modules in the mobilecommunications module 150 and at least some modules in the processor 110may be disposed in a same component.

The modem processor may include a modulator and a demodulator. Themodulator is configured to modulate a to-be-sent low-frequency basebandsignal into an intermediate/high-frequency signal. The demodulator isconfigured to demodulate a received electromagnetic wave signal into alow-frequency baseband signal. Then, the demodulator transmits thelow-frequency baseband signal obtained through demodulation to thebaseband processor for processing. The low-frequency baseband signal isprocessed by the baseband processor and then transferred to theapplication processor. The application processor outputs a sound signalby using an audio device (which is not limited to the speaker 170A, thereceiver 170B, or the like), or displays an image or a video by usingthe display 194. In some embodiments, the modem processor may be anindependent component. In some other embodiments, the modem processormay be independent of the processor 110, and is disposed in a samecomponent as the mobile communications module 150 or another functionalmodule.

The wireless communications module 160 may provide a wirelesscommunications solution that is applied to the electronic device 10 andthat includes a wireless local area network (WLAN) (for example, awireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigationsatellite system (GNSS), frequency modulation (FM), a near fieldcommunication (NFC) technology, an infrared (IR) technology, and thelike. The wireless communications module 160 may be one or morecomponents integrating at least one communications processor module. Thewireless communications module 160 receives an electromagnetic wave byusing the antenna 2, performs frequency modulation and filtering on theelectromagnetic wave signal, and sends a processed signal to theprocessor 110. The wireless communications module 160 may furtherreceive a to-be-sent signal from the processor 110, perform frequencymodulation and amplification on the signal, and convert the signal intoan electromagnetic wave for radiation by using the antenna 2.

In some embodiments, the antenna 1 of the electronic device 10 iscoupled to the mobile communications module 150, and the antenna 2 iscoupled to the wireless communications module 160, so that theelectronic device 10 may communicate with a network and another deviceby using a wireless communications technology. The wirelesscommunications technology may include global system for mobilecommunications (GSM), general packet radio service (GPRS), code divisionmultiple access (CDMA), wideband code division multiple access (WCDMA),time division-code division multiple access (TD-SCDMA), long termevolution (LTE), BT, GNSS, WLAN, NFC, FM, IR, and/or other technologies.The GNSS may include a global positioning system (GPS), a globalnavigation satellite system (GLONASS), a beidou navigation satellitesystem (BDS), a quasi-zenith satellite system (QZSS), and/or asatellite-based enhancement system (SBAS).

The electronic device 10 implements a display function by using the GPU,the display 194, the application processor, and the like. The GPU is amicroprocessor for image processing, and is connected to the display 194and the application processor. The GPU is configured to performmathematical and geometric calculation for graphics rendering. Theprocessor 110 may include one or more GPUs that execute programinstructions to generate or change display information.

The display 194 is configured to display an image, a video, and thelike. The display 194 includes a display panel. The display panel mayuse a liquid crystal display (LCD), an organic light-emitting diode(OLED), an active-matrix organic light-emitting diode (AMOLED), aflexible light-emitting diode (FLED), a mini-led, a micro-led, amicro-oLed, a quantum dot light-emitting diode (QLED), or the like. Insome embodiments, the electronic device 10 may include one or N displays194. N is a positive integer greater than 1.

The electronic device 10 may implement a photographing function by usingthe ISP, the camera 193, the video codec, the GPU, the display 194, theapplication processor, and the like.

The ISP is configured to process data fed back by the camera 193. Forexample, during photographing, when a shutter is opened, light istransferred to a photosensitive element of the camera through a lens, anoptical signal is converted into an electrical signal, and thephotosensitive element of the camera transfers the electrical signal tothe ISP for processing, to convert the electrical signal into an imagevisible to a naked eye. The ISP may further optimize an algorithm fornoise, brightness, and complexion of an image. The ISP may furtheroptimize parameters such as an exposure and a color temperature of aphotographed scene. In some embodiments, the ISP may be disposed in thecamera 193.

The camera 193 is configured to capture a static image or a video. Anoptical image of an object is generated through the lens, and projectedto the photosensitive element. The photosensitive element may be acharge-coupled device (CCD) or a complementary metal-oxide-semiconductor(CMOS) phototransistor. The photosensitive element converts the opticalsignal into an electrical signal, and then transfers the electricalsignal to the ISP to convert the electrical signal into a digital imagesignal. The ISP outputs the digital image signal to the DSP forprocessing. The DSP converts the digital image signal into an imagesignal in a standard format like RGB or YUV. In some embodiments, theelectronic device 10 may include one or N cameras 193. N is a positiveinteger greater than 1.

The digital signal processor is configured to process a digital signal.In addition to the digital image signal, the digital signal processormay further process another digital signal. For example, when theelectronic device 10 selects a frequency, the digital signal processoris configured to perform Fourier transform or the like on frequencyenergy.

The video codec is configured to compress or decompress a digital video.The electronic device 10 may support one or more video codecs. In thisway, the electronic device 10 may play or record videos in a pluralityof encoding formats, for example, moving picture experts group (MPEG)1,MPEG2, MPEG3, and MPEG4.

The NPU is a neural-network (NN) computing processor, which quicklyprocesses input information by referring to a biological neural networkstructure, for example, referring to a mode of transfer between humanbrain neurons, and may continuously perform self-learning. Applicationssuch as intelligent cognition of the electronic device 10, for example,image recognition, facial recognition, speech recognition, and textunderstanding, may be implemented by using the NPU.

The external memory interface 120 may be configured to connect to anexternal memory card, for example, a microSD card, to expand a storagecapability of the electronic device 10. The external memory cardcommunicates with the processor 110 through the external memoryinterface 120, to implement a data storage function. For example, filessuch as music and videos are stored in the external memory card.

The internal memory 121 may be configured to store computer-executableprogram code. The executable program code includes instructions. Theinternal memory 121 may include a program storage area and a datastorage area. The program storage area may store an operating system, anapplication required by at least one function (for example, a soundplaying function or an image playing function), and the like. The datastorage area may store data (such as audio data and an address book)created during use of the electronic device 10, and the like. Inaddition, the internal memory 121 may include a high-speed random accessmemory, and may further include a non-volatile memory, for example, atleast one disk storage device, a flash memory device, or a universalflash storage (UFS). The processor 110 executes various functionapplications and data processing of the electronic device 10 by runningthe instructions stored in the internal memory 121 and/or theinstructions stored in the memory disposed in the processor.

The electronic device 10 may implement an audio function, for example,music playing or recording, by using the audio module 170, the speaker170A, the receiver 170B, the microphone 170C, the headset interface170D, the application processor, and the like.

The audio module 170 is configured to convert digital audio informationinto an analog audio signal for output, and convert an analog audioinput into a digital audio signal. The audio module 170 may be furtherconfigured to encode and decode audio signals. In some embodiments, theaudio module 170 may be disposed in the processor 110, or somefunctional modules of the audio module 170 are disposed in the processor110.

The speaker 170A, also referred to as a “loudspeaker”, is configured toconvert an electrical audio signal into a sound signal. With theelectronic device 10, the user may listen to music or listen to ahands-free call by using the speaker 170A.

The receiver 170B, also referred to as a “phone receiver”, is configuredto convert an electrical audio signal into a sound signal. With theelectronic device 10, when the user answers a call or receives a voicemessage, the receiver 170B may be placed close to a human ear to listento a voice.

The microphone 170C, also referred to as a “mic” or a “soundtransmitter”, is configured to convert a sound signal into an electricalsignal. When making a call or sending a voice message, the user may makea voice with the mouth near the microphone 170C, so that a sound signalis input to the microphone 170C. At least one microphone 170C may bedisposed in the electronic device 10. In some other embodiments, twomicrophones 170C may be disposed in the electronic device 10, to collecta sound signal and implement a noise reduction function. In some otherembodiments, alternatively, three, four, or more microphones 170C may bedisposed in the electronic device 10, to collect a sound signal, reducenoise, recognize a sound source to implement a directional soundrecording function, and the like.

The headset interface 170D is configured to connect to a wired headset.The headset interface 170D may be a USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface or a UScellular telecommunications industry association (CTIA) standardinterface.

The pressure sensor 180A is configured to sense a pressure signal, andmay convert the pressure signal into an electrical signal. In someembodiments, the pressure sensor 180A may be disposed on the display194. There are many types of pressure sensors 180A, for example, aresistive pressure sensor, an inductive pressure sensor, and acapacitive pressure sensor. The capacitive pressure sensor may includeat least two parallel plates having a conductive material. When a forceis applied to the pressure sensor 180A, capacitance between electrodeschanges. The electronic device 10 determines pressure intensity based onthe change of the capacitance. When a touch operation is applied to thedisplay 194, the electronic device detects touch operation intensitybased on the pressure sensor 180A. The electronic device may alsocalculate a touch position based on a signal detected by the pressuresensor 180A. In some embodiments, touch operations that are applied to asame touch position but have different touch operation intensity maycorrespond to different operation instructions. For example, when atouch operation with touch operation intensity less than a firstpressure threshold is applied to an SMS message application icon, aninstruction for viewing an SMS message is executed. When a touchoperation with touch operation intensity greater than or equal to thefirst pressure threshold is applied to the SMS message application icon,an instruction for creating a new SMS message is executed.

The gyroscope sensor 180B may be configured to determine a motionposture of the electronic device 10. In some embodiments, angularvelocities of the electronic device 10 around three axes (namely, x, y,and z axes) may be determined by using the gyroscope sensor 180B. Thegyroscope sensor 180B may be configured for image stabilization duringphotographing. For example, when the shutter is pressed, the gyroscopesensor 180B detects an angle at which the electronic device 10 jitters,and calculates, based on the angle, a distance for which a lens moduleneeds to compensate, so that a lens offsets the jitter of the electronicdevice 10 through reverse motion, to implement image stabilization. Thegyroscope sensor 180B may be further used in navigation and motionsensing game scenarios.

The acceleration sensor 180E may detect magnitude of accelerations indifferent directions (generally on three axes) of the electronic device10, may detect a magnitude and a direction of gravity when theelectronic device 10 is stationary, and may be further configured torecognize a posture of the electronic device, which is applied tolandscape-portrait switching, a pedometer, or other applications.

The distance sensor 180F is configured to measure a distance. Theelectronic device 10 may measure the distance by using infrared or laserlight. In some embodiments, for a photographed scene, the electronicdevice 10 may measure a distance by using the distance sensor 180F, toimplement fast focusing.

The touch sensor 180K is also referred to as a “touch component”. Thetouch sensor 180K may be disposed on the display 194. The touch sensor180K and the display 194 constitute a touchscreen, which is alsoreferred to as a “touch control screen”. The touch sensor 180K isconfigured to detect a touch operation performed on or near the touchsensor. The touch sensor may transfer the detected touch operation tothe application processor to determine a touch event type. A visualoutput related to the touch operation may be provided by using thedisplay 194. In some other embodiments, the touch sensor 180K mayalternatively be disposed on a surface of the electronic device 10 andat a position different from that of the display 194.

The button 190 includes a power button, a volume button, and the like.The button 190 may be a mechanical button, or may be a touch button. Theelectronic device 10 may receive a button input, generate a buttonsignal input that is related to user setting and function control of theelectronic device 10.

The motor 191 may generate a vibration prompt. The motor 191 may beconfigured to generate an incoming call vibration prompt and a touchvibration feedback. For example, touch operations applied to differentapplications (for example, photographing and audio playing) maycorrespond to different vibration feedback effects. The motor 191 mayalso generate different vibration feedback effects corresponding totouch operations applied to different areas of the display 194.Different application scenarios (for example, a time reminder,information receiving, an alarm clock, and a game) may also correspondto different vibration feedback effects. The touch vibration feedbackeffect may also be customized.

The indicator 192 may be an indicator light, and may be configured toindicate a charging status and a battery level change, and may also beconfigured to indicate a message, a missed call, a notification, and thelike.

The foregoing descriptions are merely specific implementations of thisapplication, but are not intended to limit the protection scope of thisapplication. Any variation or replacement readily figured out by aperson skilled in the art within the technical scope disclosed in thisapplication shall fall within the protection scope of this application.Therefore, the protection scope of this application shall be subject tothe protection scope of the claims.

1. An image calibration method, comprising: obtaining a pupil positionoffset of a user and a visual acuity of the user, wherein the pupilposition offset of the user is an offset of a pupil of the user relativeto an optical axis; determining, based on the pupil position offset ofthe user and the visual acuity of the user, a first image center pointoffset corresponding to the pupil position offset of the user and thevisual acuity of the user; and adjusting a center point of ato-be-displayed image based on the first image center point offset and apupil shift direction.
 2. The method according to claim 1, wherein thedetermining, based on the pupil position offset of the user and thevisual acuity of the user, the first image center point offsetcorresponding to the pupil position offset of the user and the visualacuity of the user comprises: determining the first image center pointoffset based on the pupil position offset of the user, the visual acuityof the user, and a first mapping relationship, wherein the first mappingrelationship indicates a correspondence between a pupil position offset,a visual acuity, and an image center point offset.
 3. The methodaccording to claim 1, wherein the adjusting the center point of theto-be-displayed image based on the first image center point offset andthe pupil shift direction comprises: moving the center point of theto-be-displayed image toward the pupil shift direction by a firstdistance, wherein the first distance is the same as the first imagecenter point offset, and scene data is configured to be renderedaccording to an instruction input by the user or a default playing orderto generate a corresponding image.
 4. The method according to claim 1,wherein the adjusting the center point of the to-be-displayed imagebased on the first image center point offset and the pupil shiftdirection comprises: adjusting a displacement parameter value based onthe first image center point offset, so that the center point of theto-be-displayed image is moved toward the pupil shift direction by afirst distance, wherein the first distance is the same as the firstimage center point offset, and scene data is configured to be renderedaccording to an instruction input by the user or a default playing orderto generate a corresponding image.
 5. An image calibration method,comprising: obtaining a pupil position offset of a user and a visualacuity of the user, wherein the pupil position offset of the user is anoffset of a pupil of the user relative to an optical axis; determining,based on the pupil position offset of the user and the visual acuity ofthe user, a first distortion correction parameter corresponding to thepupil position offset of the user and the visual acuity of the user; andcorrecting a position of each pixel on a to-be-displayed image based onthe first distortion correction parameter.
 6. The method according toclaim 5, wherein the determining, based on the pupil position offset ofthe user and the visual acuity of the user, the first distortioncorrection parameter corresponding to the pupil position offset of theuser and the visual acuity of the user comprises: determining the firstdistortion correction parameter based on the pupil position offset ofthe user, the visual acuity of the user, and a second mappingrelationship, wherein the second mapping relationship indicates acorrespondence between a pupil position offset, a visual acuity, and adistortion correction parameter.
 7. The method according to claim 5,wherein the first distortion correction parameter comprises a distortioncorrection parameter corresponding to a red subpixel, a distortioncorrection parameter corresponding to a green subpixel, and a distortioncorrection parameter corresponding to a blue subpixel; and wherein thecorrecting the position of each pixel on the to-be-displayed image basedon the first distortion correction parameter comprises: correcting aposition of each red subpixel on the to-be-displayed image based on thedistortion correction parameter corresponding to the red subpixel;correcting a position of each green subpixel on the to-be-displayedimage based on the distortion correction parameter corresponding to thegreen subpixel; and correcting a position of each blue subpixel on theto-be-displayed image based on the distortion correction parametercorresponding to the blue subpixel. 8.-15. (canceled)
 16. An electronicdevice, comprising: a camera configured to shoot an eye image; and aprocessor configured to: obtain a pupil position offset of a user and avisual acuity of the user in the eye image, wherein the pupil positionoffset of the user is an offset of a pupil of the user relative to anoptical axis; determine, based on the pupil position offset of the userand the visual acuity of the user, a first image center point offsetcorresponding to the pupil position offset of the user and the visualacuity of the user; and adjust a center point of a to-be-displayed imagebased on the first image center point offset and a pupil shiftdirection.
 17. The electronic device according to claim 16, wherein thedetermining, based on the pupil position offset of the user and thevisual acuity of the user, the first image center point offsetcorresponding to the pupil position offset of the user and the visualacuity of the user comprises: determining the first image center pointoffset based on the pupil position offset of the user, the visual acuityof the user, and a first mapping relationship, wherein the first mappingrelationship indicates a correspondence between a pupil position offset,a visual acuity, and an image center point offset.
 18. The electronicdevice according to claim 16, wherein the adjusting the center point ofthe to-be-displayed image based on the first image center point offsetand the pupil shift direction comprises: moving the center point of theto-be-displayed image toward the pupil shift direction by a firstdistance, wherein the first distance is the same as the first imagecenter point offset, and scene data is configured to be renderedaccording to an instruction input by the user or a default playing orderto generate a corresponding image.
 19. The electronic device accordingto claim 16, wherein the adjusting the center point of theto-be-displayed image based on the first image center point offset andthe pupil shift direction comprises: adjusting a displacement parametervalue based on the first image center point offset, so that the centerpoint of the to-be-displayed image is moved toward the pupil shiftdirection by a first distance, wherein the first distance is the same asthe first image center point offset, and scene data is configured to berendered according to an instruction input by the user or a defaultplaying order to generate a corresponding image.